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The Invariance Principle for 
Dependent Random Variables 
By 
Patrick Paul Billingsley 


(Abstract) 


In this paper the Erdos-Kac invariance principle, as 
generalized by Donsker (Mem. Am. Math. Soc., no. 6 (1951),) 
is extended to the dependent case. Let C be the space of 
functions continuous on the closed unit interval, with the uni- 
form topology. If { X } is a sequence of random variables 
ona probability space (£1, Q3 » F), let P, be that element 

l1 . «1 


of C which is linear on each of the intervals ((j-l)n ,jn§) 


and satisfies p90) = 0 and p (in) Smee ee for 


jzl,...,n. Thus PL is a (measurable) mapping of £L into C. 
Suppose there exists a sequence { an } of positive constants 

such that if the measure P_ is defined by P(A) = Pla p, € A} 
for measurable subsets A of C, then te converges weakly 
to Wiener measure. If this is true we say that the invariance 
principle holds for { x } . Donsker has shown that the in- 
variance principle holds if { xX, } is independent and stationary 
and X, has zero mean and unit variance. Inthe present paper 


I 


we prove, after some measure-theoretic preliminaries, that the 





invariance principle holds under each of the following condi- 
tions. (i) x = f(x), where f is a function on the state space 
of a discrete Markov process {x} satisfying Doeblin's con- 
dition. (ii) { x } is m-dependent. (iii) { X } isa discrete 
linear process with medependent residuals. (iv) x, is l or 0O 
according as a recurrent event occurs or not atthe nth ofa 
sequence of trials. In each of these four cases the additional 
assumptions under which the invariance principle is proved 

are essentially those under which the corresponding central 


limit theorem has been proved. 





CONTENTS 


10; (“Gintrodtictions”. «G0; . . 2. + a. ee eee se aa 
§1. Convergence of measures on the space of continuous 
EUPCWROMMEED 5 5. sie so 9 «6 6 6 enamel re 
§2. The existence of Wiener measure .......24-e+20e.2 3V 
§3. A general invariance principle .........«-e«.«-s+ 30 
§4. The invariance principle for Markov processes. .... 43 
§5. The invariance principle for m-dependent random 
Merle. gg tt kt 8 ee sw + ee 6 + oe 
§6. The invariance principle for discrete linear processes 
Wriiwie=-dependent Teaiduals , . . . «s 6 sss s 6 2 6 5 Oe 
§7, The invariance principle for recurrent events ..... 74 
SI ee sw 8 


SUSLAC 0350 a a a er mmr nL ),', 





. Ome i nbrodwetion. 


In [9]* Erdds and Kac introduced a new method for proving 
weak limit theorems for functions of the partial sums of an inde- 
pendent sequence of random variables. Their method consisted 
in showing first that the limiting distribution is independent of 
the particular sequence and then computing this distribution for 


some convenient sequence. Let 


(0.1) xX) X,, 


be an independent sequence of identically distributed random 
variables with zero means and unit variances. Let 


S, = X, t+ -:* + X,. Erdos and Kac showed that the limiting dis- 


k j k 


tribution of 


(0.2) n7/2 max 5S 
k<n 


k 

(along with several other functions of Spree S,) is independent 
of the distribution function common to the variables in the sequence 
{X,}. Since the limiting distribution of (0.2) in the Bernoulli 
case was well known, this argument gave the limiting distribution 
*Numbers in brackets refer to the bibliography at the end of the 
paper. The expression ' Theorem i.j' refers to the jth theorem 


of §i , while ''Theorem A.j'' refers to the jth theorem of the ap~ 
pendix. 





of (0.2) under quite general conditions. There followed 
several papers (cf., e.g., Erdds and Kac [10] and Mark [17] ) 
in which this argument, known as the invariance principle, was 
applied to various other functions of the partial sums Sp. All 
of these results were subsequently subsumed under a general 
theorem due to Donsker [6]. 

Donsker's result runs essentially as follows. Let C be 
the space of functions x (t) continuous on the closed unit interval, 
with the uniform topology, and let W be Wiener measure on C. 
Let {Xn} be a sequence of random variables on some probability 
measure space (v, @ , P). Let p, be that element of C which 
is linear on each of the intervals ( (j =\) ai jn} Ned is Wt 
and satisfies pp (in) = Si for’y=1, ..., W.eand pp (0) = 0. That 


is, let p, be the random function defined by 


(0.3) Palt)=S,_)+ (nt -j+1X;. (jel)n <teune, Bal, ....0n 


where Sg = 0. Thus p,, is a (measurable) mapping of SY into C. 
Let f be any function on C which is continuous in the uniform top- 
ology at almost all (W-measure) points of C. Donsker showed 

that if (0.1) is an independent sequence of random variables which 


are identically distributed with zero mean and unit variance, then 


ra = : 
oe — i _ wii — 
——y 





lim ei '/ Zoey) = W{x: f{(x)<a} 


Ni > co 


at continuity points a of the function W{x: f(x)<a}. If, for 


example, f(x) = max x(t), then f satisfies the above conditions 


tsi 


and t(n7!/2 Py) = n7l/2 et S,, so that 
on 


0 if a 


lim r{nl/2 max Si.<a} = W{x: max x(t)}=<° a 
N—> co Ksn ostsi i out” /2.., 
e u 


0 


2 
Tr vB Gee oa 03 
where the right hand equality can be established by any one of a 

number of methods. See [6] for other functions f which lead to 
interesting limit theorems. 


It should be pointed out that in place of the ''random polygon’: 


Py, defined by (0.3), Donsker actually worked with the ' random 


step-function' with value Sj throughout the interval 
((j Sais jn7}] . There is of course no essential difference 
between the two methods. 

There is another way of stating Donsker's result. Let © 
be the Borel field generated by the open (uniform topology) sub- 
sets of C. Suppose there exists a sequence {a,} of positive 
constants such that if P, is a measure defined by setting 
P(A) = P{az} Pr EA} for AE ©, then P, converges weakly 
to W. When this is true we say that the invariance principle 


holds for the sequence {X,} with norming factors a,. Then 


ca sme me i, el a, ea 
“ae Mone) een ap 
_— = --o-m = Se a” 


Gg == — = 





(cf. Theorem 1.1 below) Donsker's result says that the invariance 


1/2 


principle holds, with norming factors n , provided { xe } is an 
independent stationary sequence with E{X,} = 0 and E{x4} wil 
The assumption that {x} 18 Stationary is relatively unimportant. 
It is the purpose of the present paper to replace the assumption of 
independence by various weaker hypotheses. 

The plan of the paper is as follows. The central results are 
contained in §§4 through 7, those of §§1 through 3 being prelim- 
inary. These first three sections are devoted to an account of the 
theory of weak convergence of probability measures on Ce (§1), 
an alternative proof of the existence of Wiener measure (§2) and 
a general invariance principle (§3). 

In §3 we sort out those steps in the proof of Donsker's 
theorem (Theorem 1 of [6]) which depend upon the assumption 
of independence and state them as the hypotheses of Theorem 3.1. 
This theorem then gives a set of conditions on the sequence {X,} 
which insures that the invariance principle holds with a suitable 
sequence of norming factors. While these conditions are not very 
pleasing, they can be verified for those sequences {X,} of great- 
est interest. 

Theorems 1.1 and 1.3 are preliminary to §3. Theorem 1.1 


gives several sets of conditions equivalent to weak convergence. 





These conditions, with the possible exception of (ii), are well 
known. Theorem 1.3, oh which Theorem 3.1 depends, gives 
a simple criterion for weak convergence of probability measures 
on © interms of the convergence of the measures of sets of 


the form 


{a ae UC SMB (Ge YS eee 


where c is a positive integer and a B; arbitrary real numbers. 


j ’ 
This theorem is a Slight generalization of one due to Donsker. 
The proof differs from his in that several arguments ‘of the 
Riemann approximation type’ are eliminated, which elimination 
is made possible by condition (ii) of Theorem 1.1. Theorem 1.4, 
which is essential to the considerations of §4, is the anologue for 
distributions on C of a well-known limit theorem for distributions 
on the real line. Its proof depends on condition (ii) of Theorem 1.1. 
Theorem 1.2 and §2 are a side issue. Theorem 1.2 isa 
result, announced by Prohorov [19], on the weak compactness of 
measures on © . Prohorov has used this theorem to give an 
elegant proof of the invariance principle in the independent case, 
but his method seems difficult to apply in those cases to which 


the present paper is devoted. His result is really an existence 


theorem, andin §3 we use it to prove the existence of Wiener 





measure. This method of proving the existence of a stochastic 
process is of course not very general (cf. [8, Ch. II] for a gen- 
eral approach) but an alternative proof of this important theorem 
iS interesting. 

In §§4 through 7, Theorem 3.1 is specialized in various 
ways. In §4 the invariance principle is proved for sequences 
{f(x,)} , where f is a function defined on the state space of a 
discrete Markov process {x,,} satisfying Doeblin's hypothesis. 
The conditions under which this result is obtained are identical 
with those under which the central limit theorem for such processes 
is proved in [8]. 

In §5 we prove the invariance principle for m-dependent 
sequences of random variables. The best central limit theorems 
for such sequences are due to Marsaglia [18], and the conditions 
under which the theorems of §5 are proved are essentially those 
of his central limit theorems. Donsker's original theorem follows 
from the results of §5. 

§6 treats of discrete linear processes with m-dependent 
residuals, processes which arise in the analysis of time series. 
Here we prove the invariance principle under conditions only 
slightly stronger than those assumed by Diananda [5] in his proof 


of the central limit theorem for such processes. 


i 


- ee 





Finally, in §7 we prove the invariance principle for the 
number of occurences of a recurrent event. Here we assume 
that the recurrence time has a finite second moment. 

In the appendix we prove several limit theorems for c- 
dimensional distribution functions. These theorems are all 
routine extensions of results well known for the case c=]. 

It is doubtful that the results of §§4 through 7 can be 
substantially improved using present methods, since in each 
case the invariance principle is proved under conditions virtually 
the same as those under which the central limit theorem has been 
proved. It is possible tc prove the invariance principle in cases 
other than those considered here. One can, for example, prove 
it for martingales, as Lévy [16] has the central limit theorem, 
or under the assumptions of Bernstein's ''lemme fondamental"’ 
[2]. Although no applications have been essayed, the cases 
treated in §§4 through 7 are those of greatest interest for the 


applications. 





§l. Convergence of measures on the space of continuous functions. 


In this section ie prove two useful theorems on the convergence 
of probability measures on the space of continuous functions. 

Consider first anarbitrary metric space %© with metric Kz : 
In what follows we will be interested in the cases in which 2€ is 
either the space of continuous functions or a Euclidean space. Let 
G3 be the collection of Borel sets, that is, the Borel field gener- 
ated by the open sets. If P,, P are probability measures on Q3 : 


we say that P, converges weakly to P (in symbols, P, => P) if 


i f dP, —> { f dP 
eS ES 
for all bounded continuous functions f. 
Theorem 1,1 gives several convenient sets of conditions 
equivalent to weak convergence. For its proof we require the 


following variation on Urysohn's lemma. 


Lemma 1.1. If A and B are closed sets with Pl, B)>0O, 
then there exists a function f(x) whichis 1 on A, O on B, 


everywhere between O and 1, and uniformly continuous on X% . 


Proof: We may of course assume that A and B are non-empty. 


With the exception of uniform continuity, it is clear that the function 





CN ees 


ls p(x, B) ied A) 





has the required properties. To prove that f is uniformly 


continuous observe first that (cf. {1, p.57]) 


|p ee, en) “PY A)| < aan 


and 
(cia i it i aaa 


From these inequalities it follows that 


(x, B) - (y, B) 


| E(x) = fly) < | 
P(x, B) + lx, A) 


1 l 
Ae | p% DN ae A) — Py: B) + Pty, A) | 
[- (y, A) ( Be 2), ee (x, B) + ( la (x, A) 
7 ; * PO BY+ Aya) fe» Bt ee 


3 
< pas) een 


Hence f is uniformly continuous on 3% . 
In what follows we denote the closure, interior and boundary 


ofaset A by A, A’ and A, respectively. If P is a probability 
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measure on %€ and f is a measurable function then P{x: f(x)<a} 


is a function of a which we call the Pedistribution function of f. 


Theorem 1.1. The following statements are equivalent. 
we PL SSP. 


(ii) ( fie. —> | f dP for bounded uniformly continuous 


“3 = functions f. 


(iii) P(A)>lim sup P,(A) for closed sets A. 
n—— >» co 


(iv) P(A)=lim P(A) for sets A €@ such that P(A)=0. 


nh == oo 


(v) For any function f which is continuous except on a set of 
Pemeasure zero, the P,-distribution function of f converges 
to the P-distribution function of f at each continuity point of 


the latter. 


Proof. We will prove in turn the implications (i)—> (ii) — (iii) 


(iv) —>(v)—>(i) . The implication (i)—*(ii) is trivial. 


(ii)— (iii). Suppose that A is closedand €>0 given. We 
may assume that A is neither the empty set nor the whole space. 
For 8 >0O let U, = {x?: 2 (x, A)< §}. Then U, is Open 

and Us | A as § \ QO, since A is closed. Hence there exists 


a § such that = 0 - A)< €, Clearly CO oe S> 0. 





- 10a -« 


Therefore, by Lemma 1.1, there exists a uniformly continuous 


mometiron ff" which is] son’ A 730 on 2 - U, and everywhere 


between O and 1. Now 


f f ap, —> | f dP 
oe J 


by (ii), and 


( { dey 2 aeeet | 
3€ 


while 





- |] -« 


{ f dP < P(A)+ P(U,-A) < P(A)+ E. 
x 


From these three relations it follows that 


lim sup P(A) pay © 


n — =» oo 


Since & is arbitrary, (iii) follows. 

(iii)—> (iv). Suppose that P(A) =0. Then 

(er) P(A) = P(A) > lim sup P_(A) > lim sup P(A). 
fh —» ao n ~~ 1 ee oo Su 


Since the boundary of 3€ -— A also has Pemeasure zero we have in 


the same way, 

clr Z) =o eewlim supalae (= =A) .. 
Nh —> oo 

Bot (lal) Sind (1.2) imply 


P(A) = lim P,(A). 
nN —> co 


(iv)—(v). Let fF and F be respectively the P,-distribution 
functions and the Pedistribution function of f and let A_ be the 
set of points at which f is discontinuous. Then 

{x: f(x})< a} C {x: f(x)<a} UA 
and 


(em) < alee A We T(x) <a}” , 








~~ 12 « 


go that the boundary of {x: f(x) < a} is contained in {x. {(x)=a} UA. 
Since P(A)=0, F(a)=F(a-0) implies that the boundary of 


{x: f(x) <a} has P-measure zero, and-hence that F,(a)-> F(a). 


(v)—>(i). Let Ens and F be the P ,7distribution functions and 
the P-distribution function of the bounded continuous function f. 
We assume that F(a) > F(a) if F is continuous at a and musi 


show that 

i * 3€ 
But this last statement is equivalent to 

M M 

\ er a) == { a dF(a), 

aM =9M 
where M is the bound of f. But (1.3) is easy to establish (cf. 
{3, p. 74]). This completes the proof of Theorem 1.1. 

We note at this point the well-known fact that if 5€ is 

cedimensional Euclidean space then equivalent to each of the cone 
ditions of Theorem 1.1 is the condition that if a and F are the 


distribution functions corresponding to P, and P respectively, 


then 


eee Flay, eee, O,) = Flay, «ees 96) 


at each point (a), sos, A.) Such that 
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F(a), ..+, Gc) = sup F(B,, «+, Bod, 
A> Pi 


i.e., at continuity points of F. 

Let C be the space of functions x(t) continuous on the 
closed unit interval, with the metric 

pix y) PP. |x(t)-y(t)] . 

Then C is a complete, separable metric space. Let (© be the 
collection of Borel sets. The Borel field © is generated by the 
sets of the form {x: x(t) < a}. That such sets belong to ce is 
obvious, and to see that they generate © it is enough to observe 


that 


{x: Gio Ns § } = () {x: [xlr)-xglr)| < § 3, 


where the intersection extends over all rationals r in the unit 
interval, 

If ty o-+, t, are fixed points in the closed unit interval, 
[x(t}), ..., x(t,)] defines, as x varies over C, a k-dimensional 


random vector on C which we denote [x, poeees X, ]. 


The first of the two theorems concerning the convergence of 
probability measures on C which we will need is due to Prohorov 


{19}. 





~ 14 « 


Theorem 1.2. Suppose that {P,,} is a sequence of probability 
measures on c. with the property that for each € > 0 there 

exists a compact set Ke such that P,(K¢ )>le € for all n., 
Then there exists a sequence {n,, } anda probability measure 


P such that P. => P as V—peo, 
» 


Proof. Let {rhs k>1} bean ordering of the rationals of the 


closed unit interval. For each n and k 


(1. 4) es Olean (i, voor 1 €5) 


is a probability measure of kedimensional Borel sets S. Let 
Bn be the distribution function corresponding to Y ieee ¢ or 
each k itis possible by Helly's theorem to find an increasing 
sequence Di } of integers and a function F(a, +e, O.) such 
that F, is everywhere between 0 and 1, is non-decreasing in 


each variable, is continuous from above and 


(1. 5) lim ye a re a, ) = FY, (a,, ae a, ) 


VY —-> oo 
at continuity points of F,. By the diagonal method it is possible 
to choose a single sequence {at so that (1.5) holds for all k 
simultaneously. 


For a fixed k let 


3 = {[x(r,), ee x(r) J: 3 Ek, i 
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Then S P is a k=dimensional Borel set with 


Vie ntSe)> or 


for all n. Since Kg is compact, S¢ is Honors From these 
facts it follows that Be must have total variation l, i.e., that it 
is the distribution function corresponding to some probability 
measure Ve . And from the remark following the proof of 


Theorem 1.1 we conclude that 
(1. 6) P. . ==> p k (vp —o ) 
‘"y 


for all k. 

We now use the measures lee to set up a measure on C 
in a way similar to that used by Kolmogorov [15] in his fundamental 
existence theorem. His theorem as such is not applicable here since 
we are working in the space of continuous functions rather than the 
space of all functions. Let + be the (finitely additive) field of sets 


of the form 
(1.7) A = {xi [x(r)), .-., x(r,)] © S}, 


where k is any integer and S is a k=dimensional Borel set. For 
such a set A put P(A) = B.S) . Since there are other represen- 
tations of A , we must show that this definition is consistent. 


Suppose then that in addition to (1.7) we have 
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(1. 8) Boe et [x(r,), ee x(r,)] Ees'}, 


where j;>k and S' is a jedimensional Borel set. From the fact 
that for any point ( s) oneleis Ss ) of k-space there exists an’ 

x E€C with [x(r,), oe x(r,)] = (=), Ber ovs S ,) it follows that 
S' = {(%,. eigen i}: (*), ees 1c € S}. From this and 


the definition (1.4) we have 

Pin) = Py lS ys weer Oy, Oy) ES) 
for all n. Hence by (1.6) 
(1.9) Pu(S) = Py fC S yp er By ee Oe Sh, 


provided the boundary of S has P ,7measure zero. But this 
clearly implies (1.9) for all S , which establishes the consistency 
of the definition of P(A). 

It is easy to show that P is a finitely additive measure cn 
+ and that P(C)=1. We now prove that P is completely addi- 
tive on F . Suppose then that {A, } is a nOn-increasing sequence 
of = sets with P(A,)>L>0 for all k. We will show that the 
A; have a nonsempty intersection. For notational convenience we 


assume that A, ig defined in terms of the first k of the {r,} : 


Ay, = { x: [x(r)), Rees x(r))] Ee S, } 2 





~ |l7 «= 


Choose € sothat 0<2€<L and let Kg be as in the hypothesis 


of the theorem, Let U, © 5S, be a compact k-dimensional Borel 


set such tnat ~ < = ; ow let 
a (5, = U,) aS, Kor 


Vie = {x: [x(r)), caer x(r))] Ee U,,} . let Ww. = Vier Vy and 


let Z, = {{x(r,), rr x(r) )j:x € Wy} . Then 
(1.10) P(S, -Z,) = PlA,- Wy) < € 


Finally, let J = {[x(r,), ince x(ri)] : oe Ses }. Now Jy. is cOMm- 
pact and hence closed. In fact, if Bon EJ, for m2>1, one can 


select x,,€K, such that [x (ry). ness; cmt) = o Since 


m e 


K, is compact there exists a sequence {m;, } of integers and an 


x€K, suchthat lim x =x. But then 
1-> oo . 
— Si “3 [x(r)), ae x(ry)] € Ji. , so that Jy. contains a limit 


point of the sequence i =? . Since J, is closed we have by (1.6), 


FS BI ee 1) S Rea > ae | bP K >le € 
(eH) Px) 2 tim sup Piya Sig? Him sup Pik, 12 


Y-P- Co 


From (1.10) and (i.11) it follows that 
Pie Sie) = ee ee eee” 


Hence Zi J, is non-empty, which implies that WK ¢ 18 non-empty. 


Since the W, form a non-increasing sequence of sets, and each is 


obviously closed, the WK have a non-empty intersection by the 


compactness of K, . Since re Gi Ay , the A. have a non-empty 





we 18 « 


intersection. Hence FP is completely additive on F ; 
Since ae generates or » P can be extended to a probability 
measure on C@. We now complete the proof of the theorem by 


showing that Fis —> FP. By Theorem 1.1 it suffices to prove that 
v 


(1.12) P(A) > lim sup Pe (A) 


Vv—> co v 


for closed sets A. Fora given € >0O let 
Sh. = {{x(r,), @e eg x(ry)] ° x € AK i } = 


Now Sy is closed for the same reason the set Jy above was. If 
B, = {x: [x(r,), —— x(r1)] = Si , itis clear that {B,} is a non- 
increasing sequence of sets of © with AK, ea pirb » Suppose on 
k° From the definition of Sy. and B. 


it is possible to find a sequence {ym} of points of AK, such 


the other hand that x € ()B 


that Ym{r,) = x(r 5) for j=1, ..., k. Since AKg is compact 
there exists a sequence {m;} of integers anda point y €AK, 


with lim von. But then x = y by continuity, and x GE AK, . 


L-> 09 


1 
Hence AK, = (\, By . Itis therefore possible to choose k so that 
(1.13) E+ P(AK , )> P(B,) = P ,(S,) ; 


Since S, is closed we have by (1.6) and Theorem 1.1, 


(1.14) P(S,) 2 lim sup Px a (S,) A 
VP — co an 





ei. 


Now by the hypothesis of the theorem, 
— c 
Pacem (Sk) 2 Pa AKG EP, (A) 


for all VP . Hence 


(1. 15) eae Pic, nO! > Ernggup Pn -~@ . 
But now (1.12) follows from (1.13), (1.14) and (1.15), since € is 
arbitrary. This completes the proof of Theorem 1.2. 

In order to apply this theorem we need a criterion for com- 
pactness of sets in ©. The following standard lemma gives a 


convenient criterion in terms of the modulus of continuity, which 


for our purposes is best defined by 
M(x, § ) = sup {|x(s)-x(t)] :s,t €[0, 1], [s-tl < 8 } 


for x ©€C and 8>0. 


Lemma 1.2. If A is a closed set such that 


(1. lo) sup |x(0)| < © 
xeA 
and 
(1.17) lim sup M(x, § ) = 0, 


§>0o xeA 


then A is compact. 
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Proof. It is in the first place easy to show that 
(1. 18) sup | x(t) < oO 
xe A 
for each t in the unit interval. Suppose {x} CA. By (1,18) and 
the diagonal procedure it is possible to find a sequence {n,,} of 


integers such that 


lim x (r) = L{r) 
Y-ro (N, 


exists for each rational r inthe unit interval, We prove that 

xO (t) is uniformly fundamental. Given €&€ >0, choose 86 sO 
Y 

that M(x, 8)<€ if &< &, and x€A. Then choose rational 


TO, eee, Ty, SO that 


<...eea 


Dee groin kc 


1 


and r,er, ,< $ for i=l, ..., k. Finally, choose N so that 
i~] o 


| x 
By 


- € 
(r) sine < 
for i=0, ..., k, provided 4,v >N. It follows immediately that 


ie, (tea, Oe < 3€ 


for all t €[0, 1], provided \,Y >N. Thus x (t) is uniformly 
v 
fundamental and hence converges uniformly to an element of C, 


which must lie in A. Therefore A is compact. 





. Gee 


In order to prove the second of the convergence theorems of 
this section we need a lemma which is a slight variation on a result 
due to Donsker [o]. If x €C and c isa positive integer, define, 


Pom =H 1, wee Ce 


A 
cr 
1A 
@) 
a 
_— 
+o ae 


apis) =inf {x(t}: (j ~1)c™ = 
bx) = sup (x(t) (i Mice <t< jcc). 

Now let 
Thx) = (aye), cee, age), BG), eee, BLO). 


Thus TT, maps C into 2c-space. It is easy to show that TT . 
is continuous. If is bounded and continuous on 2c#space, then 
oP (17 (x) is a bounded continuous functionon C. Let UW be 


the set of functions arising in this way (as c and op vary). 


Lemma 1.3. let f be a bounded uniformly continuous function on 
C. Then there exists a pair of sequences of functions {f" } and 


{f'} , all belonging to Ul , such that for all c and x, 
(1,19) f* (x) < f(x) < f(x), 

C _ C 
such that a and ie are uniformly bounded,such that 


(1, 20) lim (F(x) - fi (x) ) aU 


C —> GO 





forall x EC, 


Proof. For each c let M* be the (non-empty) set of y EC 


such that sorta y(t) < b (x) for (i =i a St ei noe | ee oe 


And define 


f(x) = inf {fly):y €MS } 


tH 


f(x) sup {fly): yEM* _— 


It is clear that f. and fC satisfy (1.19) and that they are bounded 
by the bound of £. And (1.20) follows from the uniform continuity 
on t . 

There remains only the proof that fi and fC belong to Ol 
We consider only the case of o . Let S be the set of points 


( ar se 6 ra 2c) of 2c-#space having the property that 


(1. 21) aS re j=l, sss, cc, 


and 


(1, 22) [s.. aj] [ Sap er. 7 OF, } = een 


Then S is obviously closed. If (ZF, arn 2c) ES define 
~] 
ep i. oe es o4.)=sup{ fly): Sj <ylts Bog jr (inde 


eeics, = 1, o@ @ 3 c}, 





ee 


where the set over which the supremum is extended is non-empty 
by (1.21) and (1.22). Obviously T(x) ES forall x€C, 
op (a7 (x)) = f(x), and & is bounded. ‘Suppose we prove that 
f is continuous on S. Then it is possible by Urysohn's exten-« 
sion theorem [l, p. 73] to extend Y to all of 2c~space in such 
a way that it remains bounded and continuous. Hence the proof 
will be completed if we show @F is continuous on S. 

Suppose ae > ne %5.)ES and that © >0 is given. We 


will finda § > 0 such that 


(1.23) | PCS, 06, 05.)- T(E ---, Eo) < 


provided 


(1. 24) [3 5-5 | <5 8 j= elo. 


and (& yposeee e 2¢) €S. By the uniform continuity of f there 
exists a § such that |f(x) - f(y)| < &/2 if f? (x, y)< 8 
Suppose now that ( By — < 9.) satisfies the above conditions. 


It is clearly possible to find x and y in C suchthat for j=l, ...,c, 


J = inf {x(t): G Piles <t< 5c" |} ' 
& oj = sup{x(t) : (Paice co} , 
E  = inf y(t): (Gemiiess = ica): 


N 
‘o) 
8 
— 
—— 


“1 
= 2d ’ 2 < 
& oj supfy(t): (j-l)c <t< 
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Then 


PL ee S50) 


P (arta) ) = x), 


P(E... BO) 


F P (rr i(y)) = fly), 


and it suffices to show that 
tf tt 
(1925) | f° (x) -fity)| < € 
By the definition of i there exists a z € Mt such that 
(1. 26) f"' (x) < f(z)+ €/2. 


Now let z'(t) = z(t) at points t where 


-] . al 
« < < < < 
(jel)c <t< je ; AS) ae 
let z(t) = where 
-1 . «lh 
(jel)c  <t<je ; z(t) < e : 
’ = ¥€ 
and let z'(t) Ft where 
: =} , ol 
fee=ljc 6S t Se : cet). 


ct} 


It follows that z' 2 , so that 
(1. 27) f(z <ftily), 


and from (1.24) it follows that fam z')< § , and hence by the 





5). 


choice of § : 
(1, 28) fiz) <f(z')+ E/2 . 


From (1.26), (1.27) and (1.28) we have 
14 a ty 2 3 
f(x) f.fy) : 
The symmetric inequality follows in the same way, and (1, 25) 
results. This completes the proof of the lemma. 
We come now to the second of the two convergence theorems 


of this section. For any integer c and real numbers Ape seer Ac, 


Py --+, B,, consider the set 


~] 


(1.29) a (xia, < x(t)'S B,. Heaner < .2He a1, See. 


Theorem 1,3. Suppose that for probability measures P, and P 
on @ we have Pe) —> P(E) for all sets E of the form (1.29) 


for which P(E) = 0. hep i. 7 Pp. 


Proof, Ve show first that 


(1. 30) ( ao Sa \ ale 
C C 


for any function f in Ul , Fora fixed integer c define 


p(s) = Paras) ) 


os -j 
pis) = P (7 (s)) , 
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for 2c-dimensional Borel sets S. Now if 


= 6., i= es 
wep Pye ig cya 


S = {(Z). vee Bo)? ¥,24,, 4 


then a 7(S) = E, sothat P (S)—>p(S), provided p (S) = 0. But 


this obviously implies P_ =) . Hence 
(1. 31) ( op d Pn ee ( PrP id p 
. Ree Rec 


for any bounded continuous function P . But if {(x) =f (TT (x)), 
(1.30) follows from (1.31) by a transformation of the integrals in- 
volved. Hence (1.30) holds for any function { of UW , 

Now suppose f is a bounded uniformly continuous function 
on C. By Lemma 1.3 there exists a pair of sequences : and 
{fi} of functions of Ul which are uniformly bounded and satisfy 
(1.19) and (1.20). By (1.20) and uniform boundedness we have 


: 1! a ry = 
desc) lim ( (f fr) ap QO. 


C —> 00 a 
C 


But from (1.32) and (1.19) it easily follows that (1.30) holds. And 
since (1.30) holds for any bounded uniformly continuous function Fs 
we have Pe —pP, 

In the following sections we will be concerned with "random 
polygons'', For the purposes of this discussion we define a poly- 


gon to be an element p of C with the property that the unit 





ee 


interval can be decomposed into a finite number of sub-intervals 
over each of which p is linear. Suppose (£1, QS, P) isa 
probability measure space on which random variables Xo» A): gaake: 5 x 


are defined. Associated with each point « of 1 we define a 


polygon p=p,, by the equations 


z 
iL 


(1.33) p,, (t) = (Jo nt)X, (eo) + (nt -j+1)X,(o), (j-1)n ct <jn7 : 
jal, coe, Ns, 


thatis, p is the polygon with vertices at the points Gin 


» X.). 
j! 
We will be interested in the distribution properties of p, that 
is, we will be interested in probabilities P{p@A}, where AEC. 
In order to show that P{p €A} has meaning we must show that 
{w: pP,€A }€Q9 , i.e., that the mapping w~ —> p,, is measurable. 
Since the collection of A's for which this holds forms a Borel 
field, it suffices to prove it for A's of the form A= {x:x(t)< a}. 
«] 
n 


But if A has this form, and (j =) a <t <j , then by (1.33) 


{w:p EA} = {w: (j omt}X,(obt (nt <j +X lw) < ae 


which lies in QJ since the a are measurable. 
We can use this result to set up on © a probability measure 


P' which gives unit mass to the set of polygons which are linear on 


l 


each of the intervals ((j ina jmmdatol, ..., n, andeunder 
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which the random vector (x0 >» x , x) has a pre= 


ae yr 
scribed distribution function. In factif Xo» xy ee x are 
random vectors (on some probability space ({1, Q@, P)) having 
the prescribed distribution function, and if P' is defined by 

P'(A) = P{p € A} for A € C, then P' obviously has the 
desired properties. 

A useful fact in the theory of distributions on the real line 
is that if the distributions of a sequence {X_ } of random variables 
cOnverge weakly to F , then so do the distributions of (a om ; 
provided Puy Y¥,, 79. We conclude this section with a theorem 
which plays an analogous role in the theory of distributions on C. 
The theorem and its proof obviously remain unchanged if C is 
replaced by any Banach space. 

Let {X,} and {yy} be two sequences of measurable 
functions on some probability measure space (1. ,@ , P), with 
values in C. Thatis, we assume that X (eo) EC for w € Lr 
and that {w: X (0) € A}e€@ if AE , and similarly for Yn. 


Let, 9 P! and ae be the distributions on C of X,, Y, and 


Xy, +t Y, respectively: 


BAA) =P {X,, © alle 


PuAy = Pay Gea, 


PMA) 


Pe EA}, 
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for AEC. Let U be the measure on © which places unit mass 


at the function which is identically zero. 
Theorem 1.4. If Pi, =O and Ri => U then Be eh 


Proof. It is clear that a => U is equivalent to the statement that 
(1, 34) are {7 (0, ¥,)2 € } = 0 
Nn —r 00 


forall €>0. Let f be a bounded uniformly continuous function 
on C. Given € choose & sothat |f(x)-f(y)| < € if pm yy é , 


Then 
P {| £(X,,)-£(X,+Y,)| > €} < P{e(X,, et er }—> 0 


by (1.34). Hence 


(mes) p lim (f(X,)-f(K,+¥,) =O. 


n— os 
Now by Theorem1.1, P{f(X,)<a}—>Q{x: £(x) <a} at continuity 
points of the later function. Hence by (1.35) and the above «mentioned 
fact in the theory of distributions on the real line, 

P {f(X_ +¥,)< a}—~Q{x: f(x) <a}. Since f is bounded this 


implies 


( f dPi Se { fdQ. 
KG c 


Hence by Theorem 1.1 ((ii)—>(i)) we have Pe =>'<”. 


’ 
— 
‘ 
oO 
‘ 
é é j 
— == => 


ie 7 at 


7 = 


el | —_—=—_ hein —= «§. oe 
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§2. The existence of Wiener measure. 

A fundamental problem connected with the measure- 
theoretic aspect of probability theory is that of proving the ex- 
istence of stochastic processes having specified properties. In 
this section we give a proof, based on Theorem 1.2, of the ex- 
istence of the Wiener process. 

A’probability measure P on © is called Gaussian if the 


P.«distribution of the random vector (x, 5-205 xX, ) 1S) Bomany 


t 
n 


set (t, eens. t) of points of the unit interval, a normal] distri- 
bution with zero means. For such a measure F the covariance 


function 


R(s;,t) = xo x Ve 
C s t 


is defined for all s and t inthe unit interval. Moreover R 
completely determines FP. Infact R determines the covariance 


matrix and hence the distribution of each (x, ,..., x, ), and 


: ai 


these in turn clearly determine FP . The question is,given R, 


t 


does there exist a Gaussian measure P with R as its covari- 
ance function? The following theorem gives an affirmative 
answer inthe case R(s,t) = min (s,t). The Gaussian measure 
having this covariance function is called Wiener measure, and is 


denoted here by W. It is a simple matter to show that 1f 
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toot < + *e < ta then the random variables 


are, under W , normally and independently distributed with 


zero means and variances 


Theorem 2.1]. There exists a Gaussian measure W_ such that 
iz 1) f x x, GW = min (s,t) 
Jo s t 


for all s and t in the closed unit interval. 
Proof. For each positive integer n let AL € Cc be the set of 
polygons which are linear on each of the intervals 


((j sie. 2a j = ae eal 


Let aes be a probability 
measure on © such that P(A) = 1 and such that the 


P distribution of 


(2.2) (sc35 x re en’ ¥ ) 


0 Sis oye ] 


is normal with zero means and covariance matrix ( Ne ), where 
i = 27" min (i,j). That sucha Po exists follows from the 
remarks preceding Theorem 1.4 and the fact that the matrix 


(r;, ) is positive semi-definite (in fact 





= Sc = 


a \ ae 27 2 
me. mu, = 2 ee u.) >0O.) Note that under FP 
ijro “ij ij iz) jee 3 n 
the differences 
(253) x n 7X 9% - x 
lee ee ee 


are independently and normally distributed, each with variance 27° 


We will first show that PT satisfies the hypothesis of 


Theorem 1.2. For 0< 8<1 let 6(§)=-lg§ . Then 

(2.4) a( 8) yee |S 

and 

v2 5) $6(8) 4 es, 

provided 3 5, . Givenan €, with 0< E< A! , let K, be 


the closed set 
{x: sup B( 8) M(x, & )< Ace), 
0<3s % 
where A is a positive constant to be determined later. Then K, 
is compact by Lemma 1.2, since am! ( 3 ) { Oas 8 to . We will 
prove that P (Kg) >l- €& forall n by showing that 
(2.6) P_ ex: sap B( 8) M(x, 8)> A/E}< € 
m o< §< 4 


If x € AL it follows from (2.5) that 
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sup 6(§)M(x,8)-= sup a( 8) 2 max — |x((kt1)27"} = x(k2~")| 


0<$<2” 0<8<2a" i ee 


B(27") max |x((k+1)27") - x(k27")| 
o<Kaa 
Hence, since P fA) =], since the differences (2.3) are normal 


1 


and since € <A, we have 


(2.7) Pb { sup. 6(8) M(x, §)>A/E) 
ni o<3s2°" 





<3 
EM 
D ; = a = A 
cD, Bb (B27) |x ((k#1)27) = x(k") [> 4) 
k = 0 
< = ne i ae exp(- +s . 2. | 
ab Zig 2 n é 
= , 
where 
n 
Af = supy igen’? exp(=——5— ° £5) < a 
n2i ou c n 


On the other hand from (2.4) and the fact that M(x, § ) { as 


§ , it follows that 
(2.8) sup 8($)M(x, 8) = max sup, 6 (8)M(x, 3 ) 
a0< 85 Uh Baa "-2 ga Ge” 


max 6 (2) "gee? 
2S j\<n-2 


TAS 
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But if x € AL » a moment's reflection shows that 


(2.9) M(x, 2277) 
<3 max ., sup { |x(t)-x(i2I7") |: i2) ee a4 (i+1)297" } 
04i<2°7) 
= 3 max | max , [x ((i29 + )27") - x(i227")| 


o<i<z°) o¢k<2) 
Now by a well-known inequality concerning the maximum of the 


partial sums of independent symmetric random variables 


{8, p. 106] we have, 


(2.10) p ( 3a(2 sae max Fe ote) ose Sy) } 
o< k<2! 


1A 


2 p36 (227179) |e((i41)297) : = yay > A/E } 


“y ‘ nN-j 
E 7 lge2 nejtl ] 2 
re 1c ame PSs FI ear 

mB fer , (n-j)/2 e 1e22 (n-j41)° 
From (2.8), (2.9) and (2.10) it follows that 
(2.11) a { sup 6( S)M(x, §)> Af/eée}< £. a , 

z7<235% 
where 
} ome k 
See 2k/2 441) exp (- ti -=--=-)< | 
; Pea ec 18 lg 2 (k+1) 


If we take A= A'+ A" then (2.6) follows from (2.7) and (2.11). 


We next show that 
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ae ‘ 
(2512) w xX, dp —> min (s, t) 
Suppose that s <t and let = [s2™] and al (t2"]. If xe A, then 
re Wen. -n . son 
x(s)=x(j2  )+(s20-j ACG +1)2 ) - xli,2 ))» 
s @n n =n -n 
x (t) = x(k 2 )+ (t2 ~k x(k, + 1)2 )- x(k 2 )} 
Hence, since the differences (2.3) are independent, 
_ en -n os ae 
XX dp. = \. x(j 2 ) x(k 2 )dPn = inc —S> s 


Since ae satisfies the hypothesis of Theorem 1.2, there 

exists a sequence {n,,} and a probability measure W on © such 

that Es —>w . It follows easily from the normality of the 

variables (2.2) that W is Gaussian. And (2.1) follows from (2.12) . 
The method of proof of this theorem can be applied equally 

well in other cases. It is possible to use it to prove, for example, 

the existence of the stochastic process which arises in connection 

with Doob's approach to the Kolmogorov-Smirnov theorems [7] . 


We note at this point the well-known fact [8,p. 392] that 


co 2 
ay 2 -u /2(b-a) 
W{x: per (x(t) = x(a)) > \ i Sey Ten ‘ € du 


a<t< 
It follows from this that the W-measure of the boundary of any 


set of the form (1.29) is zero. 
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3. A general invariance principle. 
In this and the following sections we will be concerned with 
proving the convergence to W of the distributions (on C) of cer- 


tain sequences of random polygons. Let X,,X ke a sequence 


jh 
of random variables ona probabaility space (f1, Q> 2). Een 


Py be the random polygon defined by 


P(t) = ai ae atin » (jel Ac aot & ae 5 & lea 


where ; = x, iil ~ and So = 0. Thus P, is the polygon 
with vertices at the points (nee Sj). As remarked at the end of §! , 
PP {P, € A} is, for A ECS , a well defined quantity. Suppose there 
exists a sequence { an } of positive constants such that if the 
measure P_ is defined by P (A) = F { amp € A} then Pio W . 
If this is true we say that the invariance principle holds for the 
sequence im } with norming factors a_. It is the purpose of 

this section to derive a general set of conditions on {x} under 
which the invariance principle holds. What the conditions lack in 
elegance they make up for in utility. In the subsequent sections we 


specialize these conditions in various ways. 


For integers c,vY and n define 


n, = [jne™'] . ie Oe cca, Co; 


n, safanl paljciay tan Nee ae 


j»u | =, . ooeeem, Wes 0S, ee 
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Hor dmy real numbersea., 8. with aeseGr ie en Co ket & 
la J~ n,f 


be the (7 set where the relations 
(oI) a. <a orp itera. <i<cn. 


are satisfied for i<r , but not for iz=r. 

Theorem 3.1. The invariance principle holds for the sequence 
{x} with norming factors a if the following two conditions are 
satisfied. 

Condition (i). For each integer c the distribution of the random 


vector 


(3.2) a (ice , Seercie neces ) 


approaches, as n-» oo , the normal distribution having zero 
® a « ~~] * * s 
means and having as covariance matrix c times the cxc identity. 


Condition (ii). For each integer c, each set 


S 
(a, see+, Oo By s+ees BO) and each > 0; 


n 
(3.3) lim lim sup 2 P(E AOS -S | ya) eee 
Vor co = 00 rer n, Tr mj, uth r n 
where the n., re corresponding to each r is defined by the re-~ 
lation 
(3.4) hn << re 


jeu = "ut 


Proof. Throughout the rest of the paper we will be dealing with sums 


isa 


of the sort appearing in (3.3). In each instance Neca: 


function of r defined by (3.4). 





_ aie 


To prove the theorem, let ET be the (© set where (3.1) 


h 
i tisfied f i = vee, N, --- ; 
is satisfied for all i=1, n. Thenf,-E. Sou ie “ 


Let E bethe C set where 
a,.< x(t) < 6, 
4= a 


° ‘ ~] . = a 
if (j - 1)c eae ror je ie weer. IEee D., be the C 


set where 


a <x(((j-l)~ tule” v 1) <8, 
for j=1,.-.., ¢c and us=l,..., YP . Further, let a be the 


S$). set where 


fom j=t,..., ¢c and u=l,..4, VW . Wintilly, bet E, ‘ Dy ¢ 


and F 
m, € 


be defined in the same way as E, D,, and Fy but 


with a and es replaced by F + &€ and 2 - € respectively. 


FOr n. <r<cn, write 
veal jiatce 
(5) RHE b= P(Be vm tolGe mess | > “ewa}) 
j,utl 
+ P(E oils, Soman EE Ha): 


j, utl 


Obviously the set in the second term of the right member of (3.5) 


is contained in 1). - an — Hence, since the ET , are disjoint, 
| 


we have 





2 Sone 


n 
ee BE ~ - t 
(E,)= 5 PUB, <1 - PUR, )+ Con | 


where e. " is the sum in (3.3). Since EB Cc FA we have 


PIF d= Sy 1S PIE) SPE). 


But 


N—» oo 


lim P(ER )= w(D., 


nh co 


a): 
by Condition (i) . Hence 


W(D ete )- eh aus nie a POE.) < PY alt P(E DS W(D, ) ' 


Letting V—-> co we have, by Condition (ii) , 
W(E . ) < lim inf P(E_)< lim sup P(E_)< W(E) . 
N-> co " nN —> co ie 


Now E, } E° as €0. Since W(E) = 0, lim P(E_) = W(E). 
n-> co n 

The proof of the invariance principle will be complete if we 
prove the following lemma, the hypothesis of which we have just 
shown to be satisfied. 
emma 37). Let AL be the set of polygons p which are linear 
on each of the intervals ((i - wen wa" and satisfy p(0)=0, 
and suppose _ is a measure on © with F&F {A ) = 1. Suppose 


further th.it 
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(3.6) P(G_) > WE) 


if E is any set of the form (1.29) and G. is the cet cf 


x EC for which 


a, < x(in”) <8, 


j 


if Bei 


Proof, Let € bea small positive rational andlet E, be the 


Shes for Bo) , sees Tien Po= WwW. 


set where 
a,.< x(t) < 
; 2 eB 
: ~] . al 
if (j-'): “+ E<t<je -€& (j=2,..., c), where 
a, £ x(t) < 6, 
Ot “lL | €, where 
a < x(t) <6. 
=] 


lec + &€ <t<1l, and where 


) 


max (a,, Ou < x(t) < min (B;, Pit 


if ic .. 9 eee Gj=1l,..., ¢c=#=1l). Analogously, 


define G_ 4 to be the set where for i=zl,...,n, 


. ol 
a.< x(1in < 6. 
cee =a 
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if Lovee e< ee See, C+ 1) ,a where 


_ =l 
7 seve 


if O< ina < ee - €, where 


. =] 
a. <x(in™) <8. 


1 1 


if l-c + € <in <1, and where 


. #l] 
max (a, a4) < xfin )< min (6;, B,4)) 


at eee Spee lave, C=-1). Simee © is 


pe iene E<in < 
rational, E, can be cast in the form (1.29) and G, 2 bears 


the same relationto E, as G_ doesto E. Then by hypothesis 
n 
(3.7) Sh a! Sarg ale) 


as n-—=o . Now E CG, while G A CE provided 
n née n 


nae E . Hence 
5 ey oes , 
i are 2), Sene 

10rsaree n and, by (3.6) andes. 7), 

(3.8) W(E._)< lim inf P (E)< lim sup P (E) < W(E). 
gE - n-» os a ~ ND» ce n ie 


Now Een EF as é€40, where WE - F)=0. Letting E-—>0 
in (3.8) we have lim P_(E)=W(E). Hence P_=>> W by 

nh 00 n n 
Theorem 1.3. This completes the proof of the lemma, and hence 


of the theorem. 
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With only slight complications Lemma 3.1 can be proved 


with W replaced by an arbitrary limiting measure. 
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4. The invariance principle for Markov processes. 

In this section we prove, using Theorem 3.1, the invari- 
ance principle for discrete Markov processes satisfying Doeblin's 
condition. We use the definitions, notations and results of 
iemch. ¥ |. 

Let X bea space of points S and let +, be a Borel 
field of subsets of X. Let {x n>1} bea Markov process 


with state space X and stationary transition probabilities 
(4.1) Big A) — Peewese etl ixe =i"). 


That is, tx} is a sequence of measurable functions from some 
probability space (12, @, F) to X, suchthat (4.1) holds, 
where the transition function p is a measurable function of S 
for a fixed A € wes and is a probability measure on Fe 


for fixed 7 . The initial distribution 7T is defined by 
TO) eee 
and the n-step transition probabilities by 
pT ey = P fx er | lex -€£ }. 
| n+l l 


The existence problems involved here are resolved in [8] 


We assume that the process satisfies the hypothesis of 


Doeblin: 
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Hypothesis (D). Thereexists a finite-valued measure Y 
on 4; , aninteger V >1 anda positive €& , such that if 


Y (A) < € then 
p(B, Ay<l-€ 


forall & € Ke. 

Note that (D) is a hypothesis on the transition function 
alone and is independent of the initial distribution wT 

It is shown in [8] how, under (D), the states $ can be 
classified according to their ergodic properties. It is shown 
further that the ergodic theorem holds if the following hypothesis 
is satisfied. 


Hypothesis (D.) ; 


(a) Hypothesis (D) is satisfied. 
(b) There is only a single ergodic set and this 
contains no cyclically moving subsets. 
That is, it is shown that if (D.) holds then there exist positive 
constants ~ and (? ; fe <1, and a (unique) stationary initial 


distribution p such that 


(n) n 
lp’ "(§ ,E) = p(E)| <7 p 


forall FE E€X,E €E Wey and n>1 . The results of this section 


will be obtained under the assumption of (D.) 


Hie’ She ‘ee 1 


eee ee = 
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In what follows the initial distribution under the assumption 
of which a probability is computed will be denoted by a subscript, 
thus: me (E). If wt =p, the stationary initial distribution, the 
subscript will be omitted. In statements involving only transition 
probabilities, e.g., (4.1), the initial distribution is irrejevant 
and the subscript will be omitted in any case. 

We state for reference three results provedin [8,p. 224]. 
Lemma 4.1. Under Hypothesis (D.); if f is a bounded (perhaps 
complex-valued) random variable, |f]| <M, on Xap 


kt+2’° °° 


sample space, then 
k 
(4.2) PRS ie eee ee: 7 MP: 


In several of the subsequent applications of this lemma f will 
be the characteristic function of a set. 
Lemma 4.2. Under Hypothesis (D.,) » let f be real-valued function 


of E , measurable ee with 
2 Zz 
E{f(x))} =0, E { (£ (x, ) ) }=e or" < oo 


Then as n—» oo , 


n 
Befa( Dette, oro oe 
j=l = 


where Ge" is a constant depending on f and on the process. 





eA 


Lemma 4.3. Under Hypothesis (Do). let £ bea real-valued 


function of § , measurable Hx , with 


2+8 
} 


E {f(x))} =0, E { |£(x,)] < oO 


for some § > 0. Thenthere isa constant a, for which 


n 
E{| z= te FS ca lle) , gl UNA ere 
j=l 


It is convenient to have available the following corollary 


of Lemma 4.1. Suppose we have positive integers Usrv. with 


(4.3) BS VU ee ee SV 


] l 2 2 Pi 1) 


Suppose further that 


(4.4) u, = Vv. 


> B 2. he rs 2 6 
i ie] — a 


Lemma 4.4. Under Hypothesis (Do) let i be a (perhaps com- 


plex-valued) random variable, with if, | <1, on a 


‘ Vv. 
J J 
sample space, for j=1,,,.,m . If (4.3) and (4.4) hold, then 
B 
(4.5) |E{£, ee enn) 1 < 2m 7 P 


Proof. The proof goes by inductionon m. The result being 


trivial for m=], assume itis true for some me«1. Then 


E{f---£ } = EB {ff Ete (| ™, = E{f,:: fj Elf} + &; 
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where € | € 2 77 E by Lemma 4.1 and the bound if, | Sul 
And now (4.5) follows from the induction hypothesis. 

We come now to the invariance principle. 
Theorem 4.1. Under Hypothesis (Do), let f bea real-valued 


function of E » Measurable 7. with 


sy 


E{f(x,)}=0, Ef{ |£(x,)| x 
for some $>0O. Then 
n 
(4.6) lim E{ (n” y2 SD £(x;) \°} = g,* 
Ni =? CO 


j=l 

exists. If g,° > 0 then the invariance principle holds for the 
sequence (f(x )} with norming factors gril? » no matter 
what the initial distribution 7T. 
Proof. That the limit (4.6) exists is simply a restatement of 
Lemma 4.2. We prove the result first under the assumption of 
stationarity and remove this restriction later. 

We must show that Conditions (i) and (ii) of Theorem 3.1 
are Satisfied. In the notation of that theorem, we must first 


prove that the distribution of the vector 


“1 i-V/2 5 65 23s ,...,58 «-S 
de My n n 


ar 


l n, 


approaches the appropriate normal distribution, where 


Sy = f(x,) +r e+ f(x, ) . Our proof of this part of the theorem 


follows [8]. Let (oc } and { BL be two sequences of positive 
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integers such that if {py} is defined by 


= 2 : -1 
(4.7) i = [ ig (n, - ny) -6 Mo +B) J, 
then 
(4.8) eines ea = Oe 
N—> oo non Aa 
4.9 1; 'n 0 
ae ees Pa ? —. 


while a Ree and ie all go to infinity. For example one can 


take pw oe and 0% B” 
Now for j =1,...,c let 
(m-1¢ dt prt of 

an = E(x nee, m=l,..., PP , 


t=(m-1 440) +1 j-l 


m Col + B) 
t = = ree . 
oe See 
iam - (ol 4+ pd te tt jJ= 


oa ti 
j, l i= pCaepoe! % ~} 
We prove that 


(4.10) p lim n- 





aaa). 


Now by Lemma 4.2, Minkowski's inequality and (4.8), 


i pal 7 +1 > 
pil? fn? z Ye al? S BVA tay) 


nel m= 1 


¢ an7/2 (p pi/2 + (a + 28)!/2) < apr? gV/2 al/2 4 (a4 2p)? 2 


where A is aconstant. This implies (4.10). 
Hence by Theorem A.4 it suffices to prove the asymptotic 


normality of the random vector 


p Y 
(4.11) ve ( Yim’tt?? ye yy. oe 
mm. ‘ m= 1 , 


Let 


c p 
A ~l1 +1/2 
BT (Gi x -U) = E {exp(i - a = J n nent } 
j=l m= 1 


be the characteristic function of (4.11) . Now the last term f(x.) 
occurring in a = and the first occurring in y Pray have 8B 
such terms in between them. And the last term of y; v and the 


first of Yiu have at least 6 others in between. Hence by 


Lemma 4. 4, 


y c 
(4.12) P (pret) = — ag E {exp (it, Yim) + € , 


where jé | < 2rcpe Pro by (4.9). Let 
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err, QoS", Cc, trite), aa p ) be independent random 
variables each having the distribution of You- By (4.12), the 
proof of Condition (i) will be complete if we show that the dis- 


tribution of the vector 


p y 
(4.13) om n7 2 ¢ i 
m= 1] = 


approaches the appropriate normal distribution. Since 


p /an->c” by (4.8), it follows that 


y 
De 2, eee ji Spe rence 


(4.14) lim E {( g7} n7i/2 
] ; Jem 


hh — co m 
Since the components of (4.13) are independent, its covariance 
matrix approaches oF. times the cxc identity matrix. By 

Theorem A.2 it suffices to show that Lyapounov's condition is 


satisfied. By Lemma 4.3 there is a constant a_ such that 
Bhp 30 


Hence, by (4.14), for n_ sufficiently large, 








y ‘ 
. E{ {or} a7l/e - p 2+ \ 
oc ll : iia - 4/2 
—— oo a < ZV —> 0 
p 
rE (a aw/2 so > 1 1+ ( 8/2) 
m=] J: 


Thus Lyapounov's condition holds. 


Having shown that Condition (i) of Theorem 3.1 is satisfied, 
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we turn to Condition (ii). Let EL cand V be as 


rej, atl! 


in that theorem. Define a sequence {B 3 of integers by 


B= Leg n | . dif ape os then 
(4.15) P(E {|s s |> en’) 
n,r™ Dy atl rio *® 
< Pike 2 alee = aia! = e n/2)2 4) 


j,utl 


1/2 
+ P{|S.,, -S_| os Gen”) /2 i), 
and we can estimate the terms onthe right separately. Now 


e nl/2/2 | Xprree eX } 


(4.16) P{|s_ -S z 


> 
j, utl +6 7 


> enV/272)4 2rpP 


< P{|s -S.ia! 2 


“Fh utl 
by Lemma 4.1 and the Markov property. And by Chebyshev's 
inequality and Lemma 4.2, 

(4.17) P{[s V2 
MF ut] S +6 =e /2} 


=] 


@2 
eR at 


- (r+B)) < DNB 6 pa; 


|A 


where A is aconstant. By (4.16), (4.17) and the defining property 


of conditional probabilities, 


1/2 
(4.18) P(E, J atls oa! = Em / om) 
earl 


< (Sf ee) 


Efoy 


To estimate the second term in (4.15) observe that 


: 1/2 
(4.19) SS. -S_|> en “/2} 


Bp 


r+f6 
= Pee: eee /2 } 
j=r+1 : 


lA 


BP {[f(x,)| > € 67 n'”/2 } 


HH 


Therefore, by (4.15), (4.18) and (4.19) , 


/2 
} 


P(E nt ls, -S_[> én ) 


j, uel 





+2 pP ple +p P{fle,)| > € Bn “V2 } 


This estimate was obtained under the assumption that 


Cet <= 2. but obviously holds in the other case as well. 


peal 


co 


Since the Ey are disjoint we have then, 


4.20) £2 PIE {|s = | oa 
(4. n,r™ Ny,utl- ip oe 


r =i 


ZA 





[A 


—— +42 vp Pang © ([£6.,) | SG ae nl/2/2 } 
f“cv 
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Now 


np P{ [E(x] > € B72 } 


Oe ee a 
n 


< (-=)°** 8 jr y 


fe E{ | £(x,) 


Hence as n-»oco the second and third terms on the right in (4.20) 
go to zero, and (3.3) follows immediately. 

We have thus proved the theorem under the assumption that 
the initial distribution is the stationary one, which assumption we 
now remove. Let 7T be any initial distribution. We show first 
of all that there exists a sequence { 6. } of integers going to 


infinity so slowly that 


(4.21) lim P_ { max |{S,[> ¢é& sue j= .0 
n—-ree TT i< p, : 
forall €>0. Foreach k select an integer m, so that 
k 
zx P_ {[£(x.)| > oe nlf }< oi 
: ahh = 
1 =1 
ifPen> m, - Clearly we can choose the m, so that my <™M,4)° 
And now let BT =k if m<n< my, Then BO goes to 
infinity and if m <n<s m4) then 
1/3 - Ber 1/5 
P_{ max |S.| > Hes = bb {|£(x.)| > kn } 
13 Pn i = | TT . 
Pi: 8 -1 
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But this implies (4.21). 
that 
(4.22) 6 =o(nl’?} 


Let P be the polygon defined by 
-] 


_ ; ae &) : 
Balt) = ea, + = (nt-jtl), QG-l)n < t< jn 


And let Pa be the polynomial defined by 


where 5 =Q. 


Pt) 


P(t) = 


I 
ns 
Ho 
t 
— 
IA 
ctr 
|A 
— 


= 
p,(6 in: ) if is 


where {6} satisfies (4.21) and (4.22). Finally, let 


Now (4.21) implies that 


= . 1 
Ph ~P,7 Ph 
) =l/2 _, : 
(4.23) lim P_ { max on P(t) Baby = 0 
hn —> co a Oo<tsi 


E>0. <Adéo 


for all 
P{ max |S,|> €n”*}< BP {[f(«,)| > 
is i 
BP 2 


Cen 


jzl,... 


It is obvious that we can also choose 6 so 


«1 ni’? } 


€ 
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so that by (4.22) 


(4.24) lim. P{ max — pea(t eee ee) 
nh —> oo os tsi = 
forall € >0. Since we have shown that the invariance principle 


holds if TT is the stationary distribution, 


p{ gi, lve 5 


: epee} WA) 


for all A €@ for which W(A)=0. By (4.24) and Theorem 1.4, 


(4.25) P {a} mee pi € A }—>Ww(A). 


1 1/2 
n P 


It is clear that the set (7) ee € A} is measurable on 


Xe41'% 42° ... Sample space. Hence by Lemma 4.1], 
Se) | ae 
ga n al €A}-P{o, n p' € A}| 


SP a. 


By (4.25) then, 


-l -1/2 
n 


Pp to, 


= pe 2) ay 4) 


But from this, (4.23) and Theorem 1.4 it follows that if 


W(A) = 0 then 


P let om - 


oe p,€ A}— > wW(A) 


We have thus proved the invariance principle with no restrictions on 7T . 
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§5. The invariance principle for m-dependent random variables. 


A sequence {X, } of random variables is said to be m-depen- 
dent if the random vectors (X,, ..., oases.) and , er me) 
are independent whenever s-=r>m. Sequences having this property 
are of interest in statistics and have been studied by various avithers 
(cf. Bernstein [2], Hoeffding and Robbins [14], Diananda [5] and 
Marsaglia [18].) In this section we prove that the invariance prin- 
ciple holds for m-dependent sequences if one or the other of two 
auxilliary conditions is satisfied. 

Then let {X_} be an m-dependent sequence of random vari- 


ables with zero means and finite variances. Let Sn = Xt. . 1 ae 


via Zz 
and a E{s"} : 


Theorem 5.1. If, for an m-dependent sequence {X,}, Bix} is 


bounded, 


(5.1) Js eng *| = O(1) 


for some constant oe > 0, and 


-(2+6) % eee 
bs = 


(Suez) lim s 2 ele 
t=] 1 


n —» 09 


for some §& > 0, then the invariance principle holds for the 


1/2 


sequence {X,} with norming factors On : 
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Proof. We first show that Condition (i) of Theorem 3,1 is 
satisfied, using the technique of Marsaglia [18] (cf. Theorem A. 4 
below). Let (n), »+e, n.) be defined as in §3, and for each pair 


(n, k) with 2m Zug Boa and each j=l, ..., ¢ define 


k-m 
es : «} 
se a i “net (i-el)ktv ' ie eas inj el) 
) d «| 
Vii BOX tikemty TSE S Ue (nen. 
ne~hs -ik+m 
) hal) i 
' = : : 
aot Youn — jtikemtv ’ 1 [k ny 
Let 
, [kn he )) : Laon 1493 
“a = x! i _ . sat) _ > ae 
n,k Y= Jol n,k ao. eed 


erty Holder "eanequa lity = k(n, -n,_j)] then 


m 
E{y! }<m 2% E{x? Pom 3, 


ae Bee ee Te a 
where B is the bound on E{x¢} . Using this inequality, a similar 


One for the case i= [k(n =n] and the fact that the Ls , are 


independent, we see that 
} (j) \2 -l1_ 2 2 
(5,3) E{(e.y) }$ (nj-n,j)k m°Bt(k+tm)"B., 


One obtains ina similar manner the inequality 
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(5. 4) Bt, 5, a -E((g 1 7} 
< o(n, =n, _j)k7'm?B + ie meee 
From (5.3) and (5.4) it follows that 
(5.5) fie» Jigaeure Tee = (ee) ENO 
a ae n,k 
(5.6) in” limesun oa JE{(S_ +S 7} 2 (20) 9°} = 0. 


k— © N—> co j jel 
Note that these two relations have been obtained without the use 
of (522). 


Now by (5.1), 
(5.7) E{(S_,.-S_)*) = jo2t+ QQ. 


where a is bounded. From this fact and (5.6) it follows that 


(5.8) lim lim Beg. g-7* E {(g be 2} =e 


k —> 00 ft —»- o9 nN, 


where the iterated limit is to be taken in the strong sense (cf. the 
appendix), 
We now show that if k is sufficiently large than the distri- 


bution of the vector 


~~ (1 =2 l _ « 
(( pias aay jroo weit Q rae ; 


where (oO) 92 = E{(g.'})2) » approaches, as n—»oo, the 
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normal distribution having zero means and having the cxc identity 
as its covariance matrix. By Theorem A,2 it suffices to show 
that Lyapounov's condition is satisfied. But for k suificiently 


large this follows from (5.7), (5.1) and the fact that 


k-m 
.|7F o eei E(x 
eee vi | 


ane 
E {ly n,_ytielikty } 


But now from (5.8) and Theorem A.5 it follows that the distribu- 


tion of 


go” n7l/2 (er, BA, ey ) 


approaches, if n-»o and then k—»o, the normal distribution 
having zero means and having = times the identity as covariance 


matrix. In order to show that the distribution of 


gt ye (See S ere Ser ) 
a esa 


approaches, as n—» oo, this same normal distribution it is enough, 


by Theorem A.4, to show that 


1 2/2 a (1) 2 (Cc) jee 


(S510) p lim plim ¢ nike ttt en kg) 


K—rco Nh —> 
a _ _ (i) (7) 
since Pan, ene + ee 


But (5.10) follows immediately 


from (5.5) and Chebyshev's inequality. 
We have thus proved Condition (i) of Theorem 3.1, and pass 


on to Condition (ii). Using all the notations of that theorem, 
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PUS yy 7Syl2 En 2/2hs B PUX 2 60'/*/em). 
Hence, 
(5.11) z PS 4 | > en/2/2}<m 2 P{|X,| > en’? jam} 
cm ORS ay Bc tix,|2ths eo 
n r=) r 
as n-~o, by (5.2). 


And (making the inessential assumption that 
rei <n. 


+] ) by Chebyshev's inequality and (5.7), 


1/2 4 4 
j, ur E'cv En 
-S is independent of E 
n. r¢+m ig 
jpaiinel 
n 


P(E, in {lS, 


1/2 
Spm! 2&2! [2a 
liner 


Hence, since 8S 


(SickZ) lim sup 


nN ——> oo 


rs 


2 
<4/cVeE, 
And now (3.3) follows from (5.11), (5.12) and 
Ere ric. _7 tis eel <aes) 
. ner 7 n. r' = 
J, ur! 
< P{|s -S_| > € n'/? 234 P(E vs -S | 
= rtm or! = nor & ut] mtr 


Sy). 





ame. 


It is possible, at the expense of complicating somewhat the 


proof of Condition (ii), to relax the condition (5.1). In particular, 


2 2 


it can be replaced by sovnge”® . 


Theorem 5.2. If {xX} is a stationary medependent sequence of 


random variables with zero means and finite variances, then the 


Wie 


invariance principle holds for i) with norming factors n vo, 


where 


2 2 e 
Te = BCs ae ea 


7 ' (j) 
Proof, Define yA a Yo ae Sao and e 


(j) 
n,k 


as in the proof of 
Theorem 5.1. It is a simple matter to show that (5.1) holds here. 
Since E{x?} is bounded it follows that (5.5) and (5.6) hold in this 
case as well. In order to establish that Condition (i) of Theorem 3,} 
holds in the present case it suffices to show that the vector (5.4) is 
asymptotically normal. But this follows immediately by Theorem A.3. 


To prove the Condition (ii) holds we proceed as before. In 


fact (5.13) and (5.12) are still valid, Finally, 
it 1/2 1/2 
ae P{|S_. 7 Sel > En /2}smnP{|X,|2 en’“/2m} 


and the right-hand side of this inequality goes to zero since 


E {xX?} < co, 
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An immediate consequence of this theorem is the original 


result of Donsker [6]. 


Theorem 5.3, If {x_} is an independent sequence of random 
variables which are identically distributed with zero mean and 


finite variance o %, then the invariance principle holds for 


1/2 


{X,,} with norming factors n‘'-@ ., 
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$6, The invariance principle for linear processes with 
m «dependent residuals. 


wet 1Y - %<j< %}be an m=-dependent process and 
{A,, t>0O} a sequence of constants such that for every integer 


n 
iy 2 AY Y; , converges in probability to a variable 
zo bat 


ee 


x; z= 0 AY Mies as n-~>#, Then we say that 


{X., -0o <j <0oo } is a discrete linear process with m-dependent 
residuals. Processes of this sort are of interest in the analysis 
of time series (cf. Diananda [5] for references to the statistical 
literature). 

Suppose that ile |; < oo} is a stationary m-dependent 
process such that wc has zero mean and finite variance and 


Suppose that 


eo 


(On) 2-9 |A,| < oO , 


Eoeemiieed 1, let i ewe 
t i- 


=) It is easy to show, 


t 


using the m-dependence and stationarity properties of {Y,} 
that 


2 2 = we ui 
al Ue -T) me pe eere in Ae, [A] ). 


agi t ten 
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Therefore 


; aN oe 
lim sup EA (aaa je eia) 


fh —> oo u> Oo 
and eG is fundamental, and hence convergent, in probvatiiity. 
n 
Thus x= ; 
random variable X; = 2a ALY , and en ~ @ Ki <ow} is 


ALY; converges in probability, as n-»~, to gome 


i 
a discrete linear process with m-dependent residuals. lit is 
trivial to show that {X;} is stationary. In [5] Diananda has 
shown that the central limit theorem holds for processes {X;} 
which arise in this way, that is, if (nigel is m-=dependent and sta- 
tionary, Yj; has finite variance and (6.1) holds. It is the pur- 
pose of this section to prove the invariance principle for such 
processes. Weare forced, however, to make a stronger 


assumption on the nature of the sequence {A,}, viz., we 


assume that 


(6.2) Ao) = cote ee 


a 


At the end of the proof we indicate some ways in which the re-~ 


quirement (6.2) can be relaxed. 


Theorem 6.1. Let {Y;, -0< j<0oo} bea stationary m-depen- 


dent process with zero means and finite variances and assume 
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that (6.2) holds. Then es, A, ee converges, agsn—>oo , 
CO 
in probability to some random variable %, = 2-0 AY oor : 


so that X, is a stationary discrete linear process with rn-depen- 


dent residuals, and the invariance principle holds for {X;} with 


norming factors nll? oe , Where 
Pim ata 2 2 a 
(6.3) idee Game <n T(E ERR > am ('h e 
1=0 A 0 Vz 0 Vv 


Proof, That {X,} exists and forms a process of the type asserted 
follows from the preceeding discussion and the fact that (6.2) 
implies (6.1). We proceed with the proof that Condition (i) 

of Theorem 3.1 holds, making use not of (6.2}, but only of its 
consequence (b.1). We will use repeatedly the inequality (easily 


established by induction on n ) 


é 2 ees? s ¢ @ 2 
ay + (an +a,) + + (ant ta) 
+ Z (atayterstay eon ( = la; | ) 
Following Diananda, define 
k-1 oo 
ae 7 Poh Ss et ‘ oH iat se et : 
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y 2 
si 
4 | eee : 
tener 
jo 


SRS 


Ap 
J 
Wi = le >. Q). _ 


1 
sou. eauall ac 
jul 
Now for k fixed and 1 varying, {P. ve is a stationary (m+k ~ij-= 
9 
dependent process with zero means and finite variances, It was 
proved in the preceeding section that Condition (i) holds for such 


processes (Theorem 5.2). Therefore the distribution of 


(6.4) (U prreee U ) 


nya, pea) Ace 


approaches, as n—»oce, the normal distribution having zero 


means and having as covariance matrix the identity multiplied 


‘Z 


by oT where 
; cn 3 2 _ al 2 
(6.5) Th ae E{U Sy yh C (Apt-+-+A, _,) d ; 
te ere ee 
where 
> m 
(6. 6) “ EY ee PS E{Y) Y,)} ; 


Val 
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To establish the second equality in (6.5) it is enough to show 


that 
(6.7) lim res P yo} = (A ferere tA 2) 

. Ni —> oo i= ion 0 k ~l : 
Let ae 7a ee " , where 

E n~k+1 
Sita ts °'s tee oo ae 

and 

at Ag Ve ey eee gee te 5 ee 


PUA ts + + + Alaa) ete a) siti ate Se 
Then 
E {¥+} = (AG to hee (A(n-k+1) +p ) é 


with A defined by (6.6) and p by 


m 
(6.8) P = -2 2 aE a eee 
If 
(6.9) A = sup | A,| ; 
tz0 


then 
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(6.10) E{y 7} <2ke Ae Eye} 
By Schwarz’ inequality, 
(ony) (E22) EM2 py 2392 ce (CE +y 
< (El? (£2) 4 V2 (y 2} y2 

By (6.10) and (6.11) , 
[[n7 Ef ( - pay - en \ (n-k+1) + P ee | 

<2k2 A2 E (y2}n™ 
which yields (6.7), and hence (6.5). From the convergence 


of the distribution of (6.4), and the limit 


@ BH = 
eae Ke om ; 


where U4 is defined by (6.3), it follows that the distribution 


of 


=} 
— Wore ie ah aes SO eeeee 


approaches, if n-»c and then k-» ~& , the normal distribution 
: ; ol on 
with zero means having as covariance matrix c times the iden- 


tity. In order to show that the distribution of 





MES 


-_—— * er oe 
] 2 i Cc Ceol 


converges, as n—»oo, to this normal distribution, it suffices 


by Theorem A.4 to prove that 


a Manic 


p lim i 


p lim 
n -—>-0o 


k —» ao 


or, by Chebyshev's inequality, that 


(6,12) lim lim sup E{V ‘ AO. , 
k—-»co N-—> oO eee 


Now, letting wen, a } » One computes 


ie 
(6.13) > 
i=n. 


~] 


co 
rl eee AA 
=k J J 


+} 


oO +2 


toe (A - 


)Y 


als ee, | a jtl-v 


fone) 
i ze 2 eee : 
V=0 


Letting 


(0.14) B= max PeibG. jaye 
| o<v<sin id 


and using the well-known fact that E{z¢} <lim sup E{Z*} if 
nN —>- 00 = 
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Z=p lim Z,, one deduces from (6.13) that 


nN — oo 


2 


(615) 9 -E{V }<injen, aT BUAL HA te, 


2 
Tie aa 
which yields (6.12). This completes the proof that Condition {i) 
holds. 

In order to prove that (3.3) holds in the present circum= 
stance, we first show that there exists a sequence {f,,} of integers 


tending to infinity in such a way that 

(6.16) lim npP{|x,| > ep al/4/2} = 0 
n-~—> co 

and 


oe tA 


ni Oo 
(a. 17) lim > Pas YL Agyyt: ; 


n—-oco Fe} v=o 


a eet 


> Bey) 2eO, 


It is easy to show that X%, has a finite second moment. Hence 


there exists an increasing sequence {m,,} such that 
ee Patsy | & fe nl/2 y a hee 


ifn2>m, . If one puts 6, =k for m, <ngmyyi than B,, goes 


to infinity and (6.16) holds. It follows from (6.2) that 
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“2 


(6.18) R, = O(n 9), 


where R, = Anal +|A +r, ~ NOx 


nt 


(6.19) P {| = Y (Aa, teee+A i> @ nl/2/4) 
(6. v=o «Vv Pv oe aie 
oo 
1/2 
< P{ a, Powe | Ras > €n’“/4} 


provided n is large enough that 


oo 
yd (a+ v)72/4 < €/4 
veo 
But 
P{IY_|| Ray > al/? (pay /4} <n (tv)? Re, 


oo Se 
so that by (6.18) the sum in (6.17) is dominated by Zz 29 (b+v) 3/2 


which goes to zero since f goes to infinity. Hence (6.17). 
We now decompose the summand in (3.3) into 


lle 1/2 
(6.20) P(E, a (Sa > en’*})< P{|S_,,-5,| 2 eni/2/2} 


1/2 
+P(ED Als, “S,+p > € a /2}) , 
iu! 
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where {p,,} satisfies (6.16) and (6.17). It follows immediately 


from stationarity and (6.16) that 


n 
(6,21) lim = P{|Sien “oule> € al/2/2} =O ». 
Nw co P= rtp a 
Let Dy ut 7* =w. Then Sa. - S.. -&+%, where 
j, utl 
E Co 
7 a (Appetit °° Fae} ce & 

and 


PRCT (a ceo Oe es )¥ 


rtwel weBel’ rt+6+l 


+. Bet Aeetieceaies )Y 


vtweBp rtpev 


Hence 


(6.22) PIE a {IS, > e nl/272}) 


oS | 
j, utl ae 


< P(E, .) ea > enl/2 yaya P{lé Paka 


where the factorization of the first term on the right is valid if 
n is large enough that Pe >m. Now by stationarity and (6.17) 
we have, 


(6.23) lim ¥ P{|&|> ent 2/4} =. 0%, 


ha co P=] 





ior 


Using the m-dependence property, one computes 


E{ 2) < 2A2Bmwt B+ (Ag tA test (A gt tA 4)? 
g-1 
+ Pay (A tooe fA 2 
“ey Vv vtwe-B 


Co 
< w(2A2Bm+( = |A,|)*), 
VvzoO 


where A and B are defined by (6.9) and (6.14). Since, in the 


notation of Theorem 3.1, w<nAev , 
a 1/2 2 

(6.24) eee ee qyliemern /4} <(const.) 16 / E% cw. 
r= | ¥ 


Pimaily, (3.3) follows from (6p20)—, {6.cl), (6.22), (6.23) and 
(6.24), completing the proof of the theorem. 
It is clear that (6.2) can be replaced by the weaker condition 


fonts). in fact 


wle 5 
n 


RewaaiOal } ; Seen0:, 


nN 


suffices, An examination of the proof shows that if there exists a 
sequence {BL} going to infinity in such a way that (6.16) and 
(6.17) hold, then the result follows. This fact can be used to 
weaken (6.2) under the assumption that & possesses some 


moment of order higher than two, 
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§7. The invariance principle for recurrent events. 


Let & be a recurrent event in the sense of Feller (cf. [i1] 


or [12]). Suppose that ¢&& is certain, let 


A> X55 awe 


be the successive recurrence times of & and let 


3, = Rt ty. 


Let Z, be 1 or O according as & occurs or not at the nth 


trial and let 


Nn = 2, +°°+ +2n 


be the number of occurrences of (9 during the first n trials. 
Assume that the recurrence times have finite mean iY and 
variance @  . In this section we prove the invariance princi- 


ple for the sequence {Z,, - p as , 


Theorem 7.1. If the recurrence times have finite mean fi 
and variance og“, then the invariance principle holds for the 


sequence {Z, - p “ty with norming factors @& p -3/2 ne 
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Froof: Feller has proved the central limit theorem for N, 
by reducing it to the central limit theorem for 5S, via the 


identity 
(7.1) {N,>k} = {S. <n}. 


Our proof that Condition (i) of Theorem 3.1 holds proceeds in 
the same way. Let ¢ ( Gps sees a.) be the normal distribution 
«1 


with zero means and covariance matrix (Ajj), where ij =C 


min (i, j). We must show that 


; ml ove 1/2e = 
(7.2) lim P IN, 73; <a n ite Aree, 


n> ow 


g(a), . an). 


Mor j=1, ..., ¢ let K = isn) be one greater than the integral 
part of ns po + ie wy -3/2 nl/2 . Then (7.2) reduces to 
(eo) im P{N < ko fads vee c} = oC ee 
"5 
By (7.1) we see that (7.3) will follow if we can prove 


Sk. ok; Tos, p 


lim {Sore aire ’ j=l, eran c} 738i). ---. 92) 
N —> 00 
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Since 


=i/2 “iy2 
j saa n 


n.e#k. a 
J 


and P (-a,, as ey oa.) = le ? (a), ane, 2 Bes |e it suffices to show that 


the distribution of 


] 
S ek ee40e S ek ) 
7 = «lfe Dey ( i] 9 


approaches @ . But this follows easily from Theorems A.3 and A.4 
and the fact that k. ~/n., y hs ° 
J J 
We need a subsidiary result for the second part of the proof, 
For each integer r let U, be the last trial before the (r+1)st 
at which © occurs, letting U, =1 if there is no occurrence of 
@& inthe first r trials. And let V, be the first trial after the 


rth at which Gs occurs. Then 


(7. 4) P{V.-U, =k} = P{X,=k}. 


For when © occurs the process begins anew (cf. [ll]) so that 


P{V.-U,=k} = 2 ee P{U,=v} P{X,=k} = P(X,=k} ; 


To prove that Condition (ii) of Theorem 3.1 holds, define 


random variables £§ by 


n,r 
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Ne if rel 
GB = 
Ar 
ns utl ees atl 
Then 
2 
=| a ) 
(7.5) P(E. _ 4 |(n an yo (N-rp yl seen  } 
igen { Se junp ic p 
Si wal ee 4/2 
p ea * baad = 
PLN, 7 ) (N. By | > Jigs } 
-1 
+ P(E a{|(N te ) 
nr el juti P 
-1 6, 2) 
- (Ng eis |e eo ). 
Now NB. aa is 0 or 1 (according as Wi cara Ls ara or not), so 


that HON, Par p) _ (Ni ym) rr po me . 


And 6 er<vV - U, , so that 


p{p. _-r> & 1/2} nl/2y 


£ 
ae 4 4 


1A 


1) 


Hence if n is large enough that > 





aie 


] 


where the limit holds because X, has a finite second moment. 
On the other hand, by the defining properties of recurrent events, 


the second term of the second member of (7.5) is equal to 


mee) P{|n_ 


ee eee 
j,uti7P rs ld ae J 


It is shown in (11, p.111] that 


E(N,) = kyo _ (ot pep “ype Sra 


and 


Var {N,}~ko*p 7? 


Hence there exists a constant A such that 


EL(N,-kp 7 )°} < Ak 


From this and Chebyshev's inequality it follows that 


| zal E 1/2 2 
P{|N tne - 6) |>sn < 4A/eé“%cY. 
ny usy7 8 bust PE an 





Hence 


h 
i7e 1) lim lim sup = P(E __ad{(N ere } 
Sane, Mica, ce, EN nr { mel judi P 


-(Ng-Bp ) se ll 5° 


Finally, Condition (ii) follows from (7.5), (7.6) and (7.7), 
completing the proof of the theorem. 

It is possible to prove this theorem by a direct extension of 
Feller's method, avoiding the use of Theorem 3.1. Thatis, itis 
possible to prove it using relation (7.1) and the invariance principle 
for {X, - p } (Theorem 5.3). While such a method is conceptually 


appealing, the details of the proof become very involved. 
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Appendix. 


In this appendix we prove some multivariate central limit 
theorems and extend to several dimensions two theorems of 
Marsaglia [18]. The proofs are straightforward but seem not 
to be available in the literature ina generality sufficient for our 
purposes, 

Let ae k=l, ...,V(n), n=l, 2,... } be an array 


of random vectors 


x (1) (c) 
Mes (Xe a eg XT ke ). 
( 


We assume that each a has mean zero and a finite variance 
9 


and that the vectors with a common first subscript are independent. 


y(n) x i) 2 


ES _ 2 
Let Pir = Ze nk *MkSae E{S.j} and let A. be the 


covariance matrix of the random vector 


~] o] 


— 1, sepeety 


S ). 


n,c n,c 


In what follows we assume that 


(A, 2) ey = (a. ) = lim AL 


oJ n> © 


exists. The matrix A is of necessity positive semi-definite, 


and for covenience we assume it to be positive definite. 





. 


The first result says in effect that if (A.2) holds and the 
Lindeberg condition holds in each of the c components then the 
central limit theorem holds. The proof reduces the problem to 
the central limit theorem in one dimension, making use of a 


technique due to Cramer and Wold [4]. 


Theorem A.1. If {A.2) holds, where the limit matrix A is 


positive definite, and 


<2 ¥¢n) 
(A. 3) lim s = 


n —> oo n, ) k=4 


(j) 2 _ 
(Ke! dP = QO 
(j) 
{1X24 | 2 € nae 
for all € >0O and j =1, ..., c, then the distribution of (A.1) 


converges to the norma] distribution having A as its covariance 


matrix and zero means. 


Proof. We must show that 


c ra 
(A.4) lim Efexp(i = t; a S  iyP= exp(-1/2 y a ) 
n—> 09 jz! 


: ee re 
J n, J Nn, J ijt 13") 
for all (ts sees to). Let the t be arbitrary but fixed throughout 


the following discussion. For notational convenience we assume 


that no - is zero, Let 


: “ly (j) 
‘ = 3% €. Ss =e xX P 
n,k jer 1 nae n,k 


It is easily seen that the second moment of mg Y 
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¢ 
a ~ 2 ain) ee: 
n tj! 1,] 1 J 

where an is the (i, j)th entry of a . Note that 3 > Oiler 


sufficiently large n, since A is positive definite. If we can 


prove that the central limit theorem holds for the array es ey 


then 
E yon) ™ 2 

(A. 5) lim E fexpdca ae > Y i} me 2 ue/e 

N—> 0 ae 
Since 

Cc 

(A. 6) i so Sf a(n) t.t. > 0 

«(0 ajes J) 
by (Ao.2), substituting (2. ‘el a. t. ty yi /2 for u in (A. 5) 


gives (A.4),in view of a well-known theorem (cf. Theorem A.5 
below). 

In order to show that the central limit theorem holds for 
the array {yy it is enough (cf., e.g., [13]) to show that 


ned) lim 5 23 Y dP = QO. 


nh 7 oo n k=] n,k 


(al 2€ s 3 


for all € >0. Now choose ree so that 


G 
eo <c Che 


.t.t.)( max ied! ‘hia , 
tycl , J 


(j) 
and let ne be the set where [xk | > og ne for some 
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j=1,..., ¢. By (A.6) and Holder's inequality we have, for 


large n, 


vin) > v(n) c - | - 
gee fo. de < ae es wee) ae 
me n, ror 
Ne ieee! n,k 


Using (A.6) again, we see that in order to prove (A. 7) it suffices 


to prove 
yn) 
(A. 8) ae ee (X wg soon 
nro Md yay A n, 
n,k 
tory = ly «es, CC. But 
y(n) = y(n) 
me > x Ui) 2) ee x i) )o dP 
n, J 2 n,k n,j é n,k 
k= { A k= 
Bids (j) 
{x 2 os, ;) 
¥Ch) ? 
eae. >> (x6) 2 ap | 
nad be] n,k 


(i) 
Mel Xk l2 Cs, ;} 


Now the first term of the right-hand member of the preceding ine 


equality goes to zero by (A.3), while the second term is dominated 


by 
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YCh) c 
a Si ae. “<> > ne (X _ BiacP . 
k= {21 k=1 
: Avge iy 
(ee as 


which goes to zero, again by (A.3). Thus we have proved (A. 8) 
and nenes the theorem. 
The multivariate version of Lyapounov's theorem now foliuws 


immediately, 


Theorem A.2. If (A.2) holds, where the limit matrix A is 
positive definite, if the xa all have finite moments of order 


2+ 86 for some §8>0, and if 


V (hn) ; 
(A.9) lim, 97>? see BO 2 9 


n-- eo ns] k=} 


for j=l, ..., c, then the distribution of (A.1) converges to the 
normal distribution having A as its covariance matrix and zero 


meens. 


Proof. Since 


~2 (j) 2 -§ a2-8 si 2+6 
or (Ke dP < §£ oh he ‘| her 
(j) 
{xX 2 és, s} 


(A.3) follows from (A,9). 
A second consequence of Theorem A.1 is the multivariate 


central limit theorem for identically distributed random variables. 
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moeorem 4.3. If the ea are identically distributed, each 
having the positive definite matrix A as its covariance matrix, 
and if Aira? (n) =o , then the distribution of (A.1) converges to 
the normal distribution having A as its covariance matrix and 


zero means. 


Proof. Let the distribution function of ee be ae where 


© 
[ dF s(x) = go . Then 


vCth) . ras 
aa = (x (3) ye ap we a dF. (x) 
Nn, ) k=] (i) n,k J 1/2 
j 
ee a {Ixl2eqv (n)} 


goes to zero, and the result follows, 
The next two theorems are trivial extensions to c dimensions 
of results of Marsaglia [18] on iterated limits. Here an iterated limit 


is used in the strong sense, i.e., lim lim a =a means 
k-> Co fh oe n,k 


lim lim sup ja-a ce ()) a eee py) oa vic) and 


K>oco fh ~—~—> w n,k 


: (j (j) 
Vise = (Vege a Vie are c-dimensional random vectors then by 


p lim p lim bo = V 


ko hr oO 


we mean 


lim lim P {|v Weer 21, ..., c) = 1 


k->°co n—>oo 


Gj) 
n,k 


for all € >O, 





a 


Theorem A.4. Let V = (V (1) Mes Pa x) and 
U = (U (1) ule) ) be random vectors such that 
n,k nk eee OO ois ' 


palim. “pein UL 12 aes 


k-»oco =n 


and 


wage? 


lim lim piv? <x,, = lee c} = G(x), ee 
K > & 1, —» 00 , J 


for all continuity points (x), Siokeae x.) of a distribution function G. 


Then 


lim lim P vi) <x» i> Wome. -5-'t c} = G(x,, eaneee el 


K-—> oo h —t go 


at continuity points of G., 


Proof. Marsaglia's proof goes over almost word for word. Let 
E = 88 >0 anda continuity point (x), s++, X,) of G be given. 


We must find integers K, Ni): N,. ee. such that 


(4.10) jpiv Gl 4 gee ae Pere c} - G(x sa x.) < ec 


j 1? e @ 
if k>K and n>N,. First choose B sothat (x,-8, ..., x. -6) 
and (x15 +B, ..., x, +8) are continuity points of G and so that 


(A.1)) O0< G(x, +B, ..., x, +B) = G(x, -8, ..., x, =6) < Ss. 


Next choose K, N,, N so that 


1’ 2? ee 6 





ey iz) 


(A.13) 


(A.14) 


(A.15) 


= 87 = 


na U uy s pee & 
j=l fi, p | 


PPEV I ca Geb wees Ce Gla coer xedl < &, 


[piv Ul <x, -6, pel, woes Ch - G(x, -B, eoes x. -B)| < 8 ; 


pPiv Oy <x, S(t aa eee tes ~ G(x) +B, rater: x +8) < 8 


provided k>K and n>N,. For sucha pair (n,k) set 


F(€) = piv +04) om oe ie |, .aaeenan 


ag) PIO) su cx +e, 10911 <6, 321... €) 


Lie) = Pv) cme WON Ise. i=1, «2. ch. 


Q(§) = piv) <x +8 ¢ j=l, eee» c} 9 


T(#) = G(x, + & : coer KO TG ). 


Then 


{| F(0) ~ T(O)| < |F(0) - H(O)| + |H(0) - L(oO)| 


The first, 


+ |{L(0) -©(0)| + | Q(0) - T(0)| 


third and fourth terms on the right are each less than & 





cane - 


by (A.12), (A.12) and (A.13) respectively. Since 


L( -8) < H(0) < L(p) 
and 
L(-8) < L(O) < L(B), 


we have 
| H(0) = L(0)| < | L(B) - L(-8)| < | Lip) - O(B)| 
+ |Q(B) ~ T(B)| + | T(B) = T(-8)| + | T(-B) - Q(-8)| 
+ |Q(-B) = L(-B)| < 58 


by (A.12), (A.15), (A.11), (A.14) and (A.12). Hence 


|F(0) - T(O)| < 8S , whichis (A,10). 


Theorem A.5, Let Lee k be random vectors such that 
9 


limo. li | (i) < Xs i eee G(x,, coe, X,) 


ke No n,k =~ 

at continuity points of G. Suppose that 
is lim Neen 
KkK-»~° n-> oo n,k 


[Clim , C. Lhen 
lim lim Pf{ Gy G) 
n,k 


< mt lL, eae, =G p eee, 
ko nF o og) ena ee * 


at continuity points of G. 





Proof, By the preceding result it suffices to show that 


plim plim (( 7 : eS Wes) = 0, 


k—» oo fh —» 00 


which is easy. 
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